Markov Chains , Coupling , Stationary Distribution
نویسنده
چکیده
In this lecture, we will introduce Markov chains and show a potential algorithmic use of Markov chains for sampling from complex distributions. For a finite state space Ω, we say a sequence of random variables (Xt) on Ω is a Markov chain if the sequence is Markovian in the following sense, for all t, all x0, . . . , xt, y ∈ Ω, we require Pr(Xt+1 = y|X0 = x0, X1 = x1, . . . , Xt = xt) = Pr(Xt+1 = y|Xt = xt).
منابع مشابه
Lecture 17 Lecturer : Anna Karlin Scribe : Eric
Coupling is a useful tool in the analysis of the mixing time of Markov chains. The basic idea is that a Markov chain that is initialized to some arbitrary distribution can be compared via coupling with another Markov chain that is initialized to the stationary distribution. The two chains then progress simultaneously, and the distance between the two chains at any time indicates how close the r...
متن کاملThe Randomness Recycler Approach to Perfect Sampling
1. The Randomness Recycler versus Markov chains At the heart of the Monte Carlo approach is the ability to sample from distributions that are in general very difficult to describe completely. For instance, the distribution might have an unknown normalizing constant which might require exponential time to compute. In these situations, in lieu of an exact approach, Markov chains are often employe...
متن کامل2006 Coupling from the Past
We saw in the last lecture how Markov chains can be useful algorithmically. If we have a probability distribution we’d like to generate random samples from, we design an ergodic Markov chain whose unique stationary distribution is the desired distribution. We then run the chain (i.e., start at an arbitrary state and evolve according to the transition matrix), until the process is at (or close t...
متن کاملConcentration inequalities for Markov processes via coupling
We obtain moment and Gaussian bounds for general Lipschitz functions evaluated along the sample path of a Markov chain. We treat Markov chains on general (possibly unbounded) state spaces via a coupling method. If the first moment of the coupling time exists, then we obtain a variance inequality. If a moment of order 1+ ǫ of the coupling time exists, then depending on the behavior of the statio...
متن کاملT - 79 . 5204 Combinatorial Models and Stochastic Algorithms
I Markov Chains and Stochastic Sampling 2 1 Markov Chains and Random Walks on Graphs . . . . . . . . . . . 2 1.1 Structure of Finite Markov Chains . . . . . . . . . . . . . 2 1.2 Existence and Uniqueness of Stationary Distribution . . . 10 1.3 Convergence of Regular Markov Chains . . . . . . . . . . 14 1.4 Transient Behaviour of General Chains . . . . . . . . . . 17 1.5 Reversible Markov Chains...
متن کامل